Skip to content

Support test automation exclusions in JUnit5 - #12941

Open
calvinbayer wants to merge 2 commits into
codex/junit4-automation-exclusionsfrom
codex/junit5-automation-exclusions
Open

calvinbayer wants to merge 2 commits into
codex/junit4-automation-exclusionsfrom
codex/junit5-automation-exclusions

Conversation

@calvinbayer

@calvinbayer calvinbayer commented Oct 19, 2026 •

Copy link
Copy Markdown

What Does This Do

Support test automation exclusions in JUnit5. Wire test declaration metadata into Datadog policy selection and reporting, preserving assigned Test Management properties.

Motivation

Allow individual tests to prevent Datadog quarantine/disabling and exclude Datadog ATR, EFD and Attempt to Fix. Developer selection and framework retries retain their existing semantics.

Additional Notes

Stacked on codex/junit4-automation-exclusions; review this PR against its base for the framework-specific change. Shared public API and policy are introduced by the first JUnit4 PR.

Validation: Module spotlessApply and instrumented JUnit5Test.test automation exclusions* passed, proving one execution, blocking failure, both annotation tags, preserved backend assignments, and no attempt-to-fix result.

@calvinbayer calvinbayer added type: feature Enhancements and improvements comp: ci visibility Continuous Integration Visibility tag: ai generated Largely based on code generated by an AI or LLM labels Oct 19, 2026
@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from 2528d3f to c943aca Compare October 19, 2026 21:02
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 58.85 57.69 $\color{red}{\blacktriangle}$ +1.16 56.55 $\color{red}{\blacktriangle}$ +2.30 78/409
agentEvpProxy 59.34 n/a n/a n/a n/a -

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from c943aca to 0b80b18 Compare October 19, 2026 21:20
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 92.06 91.39 $\color{red}{\blacktriangle}$ +0.67 91.39 $\color{red}{\blacktriangle}$ +0.67 39/204
agentless 84.84 84.36 $\color{red}{\blacktriangle}$ +0.48 82.69 $\color{red}{\blacktriangle}$ +2.15 39/204
agentlessCodeCoverage 99.88 99.00 $\color{red}{\blacktriangle}$ +0.88 97.04 $\color{red}{\blacktriangle}$ +2.84 39/204
agentlessLineCoverage 116.24 116.18 $\color{red}{\blacktriangle}$ +0.06 113.88 $\color{red}{\blacktriangle}$ +2.36 38/204

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 36.19 38.67 $\color{green}{\blacktriangledown}$ -2.48 37.91 $\color{green}{\blacktriangledown}$ -1.72 40/205
agentless 36.10 37.91 $\color{green}{\blacktriangledown}$ -1.81 37.15 $\color{green}{\blacktriangledown}$ -1.05 40/206
agentlessCodeCoverage 44.63 46.30 $\color{green}{\blacktriangledown}$ -1.67 45.38 $\color{green}{\blacktriangledown}$ -0.75 40/205
agentlessLineCoverage 54.52 57.69 $\color{green}{\blacktriangledown}$ -3.17 56.55 $\color{green}{\blacktriangledown}$ -2.03 40/205

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 21.96 22.54 $\color{green}{\blacktriangledown}$ -0.58 22.54 $\color{green}{\blacktriangledown}$ -0.58 42/239
agentless 19.32 19.59 $\color{green}{\blacktriangledown}$ -0.27 19.59 $\color{green}{\blacktriangledown}$ -0.27 41/231
agentlessCodeCoverage 20.01 20.39 $\color{green}{\blacktriangledown}$ -0.38 20.39 $\color{green}{\blacktriangledown}$ -0.38 40/231
agentlessLineCoverage 27.49 27.52 $\color{green}{\blacktriangledown}$ -0.03 27.52 $\color{green}{\blacktriangledown}$ -0.03 22/156

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 10.36 10.13 $\color{red}{\blacktriangle}$ +0.23 9.73 $\color{red}{\blacktriangle}$ +0.63 38/203

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 15.95 11.42 $\color{red}{\blacktriangle}$ +4.53 10.54 $\color{red}{\blacktriangle}$ +5.41 39/206
agentless 12.59 10.33 $\color{red}{\blacktriangle}$ +2.26 10.33 $\color{red}{\blacktriangle}$ +2.26 40/205
agentlessCodeCoverage 19.78 18.08 $\color{red}{\blacktriangle}$ +1.70 17.73 $\color{red}{\blacktriangle}$ +2.05 37/187

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.55% (-0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 308e8e1 | Docs | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 13.65 13.67 $\color{green}{\blacktriangledown}$ -0.02 13.67 $\color{green}{\blacktriangledown}$ -0.02 40/217
agentless 11.85 12.37 $\color{green}{\blacktriangledown}$ -0.52 12.37 $\color{green}{\blacktriangledown}$ -0.52 39/216
agentlessCodeCoverage 14.22 15.72 $\color{green}{\blacktriangledown}$ -1.50 15.72 $\color{green}{\blacktriangledown}$ -1.50 39/216
agentlessLineCoverage 16.99 18.08 $\color{green}{\blacktriangledown}$ -1.09 17.73 $\color{green}{\blacktriangledown}$ -0.74 38/215

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 95.37 97.04 $\color{green}{\blacktriangledown}$ -1.67 95.12 $\color{red}{\blacktriangle}$ +0.25 42/223
agentless 91.23 91.39 $\color{green}{\blacktriangledown}$ -0.16 91.39 $\color{green}{\blacktriangledown}$ -0.16 41/221
agentlessCodeCoverage 100.08 101.00 $\color{green}{\blacktriangledown}$ -0.92 101.00 $\color{green}{\blacktriangledown}$ -0.92 43/223
agentlessLineCoverage 101.93 103.04 $\color{green}{\blacktriangledown}$ -1.11 103.04 $\color{green}{\blacktriangledown}$ -1.11 49/246

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - okhttp

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 19.17 19.20 $\color{green}{\blacktriangledown}$ -0.03 19.20 $\color{green}{\blacktriangledown}$ -0.03 42/222
agentless 18.03 18.82 $\color{green}{\blacktriangledown}$ -0.79 18.82 $\color{green}{\blacktriangledown}$ -0.79 40/222
agentlessCodeCoverage 21.08 21.65 $\color{green}{\blacktriangledown}$ -0.57 22.09 $\color{green}{\blacktriangledown}$ -1.01 42/228
agentlessLineCoverage 37.25 37.91 $\color{green}{\blacktriangledown}$ -0.66 38.67 $\color{green}{\blacktriangledown}$ -1.42 41/221

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - spring_boot

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 16.24 16.04 $\color{red}{\blacktriangle}$ +0.20 16.36 $\color{green}{\blacktriangledown}$ -0.12 40/217
agentless 9.33 9.73 $\color{green}{\blacktriangledown}$ -0.40 9.92 $\color{green}{\blacktriangledown}$ -0.59 40/217
agentlessCodeCoverage 13.75 13.67 $\color{red}{\blacktriangle}$ +0.08 13.67 $\color{red}{\blacktriangle}$ +0.08 40/219
agentlessLineCoverage 22.11 22.54 $\color{green}{\blacktriangledown}$ -0.43 22.54 $\color{green}{\blacktriangledown}$ -0.43 39/216

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@dd-octo-sts

dd-octo-sts Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.03 s 13.96 s [-0.3%; +1.2%] (no difference)
startup:insecure-bank:tracing:Agent 12.88 s 12.95 s [-1.2%; +0.0%] (no difference)
startup:petclinic:appsec:Agent 17.73 s 17.67 s [-0.5%; +1.2%] (no difference)
startup:petclinic:iast:Agent 17.53 s 17.67 s [-1.8%; +0.2%] (no difference)
startup:petclinic:profiling:Agent 17.49 s 17.57 s [-1.6%; +0.7%] (no difference)
startup:petclinic:sca:Agent 17.78 s 17.74 s [-0.7%; +1.0%] (no difference)
startup:petclinic:tracing:Agent 16.62 s 16.72 s [-1.6%; +0.4%] (no difference)

Commit: 308e8e18 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sonar-java

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 27.21 8.80 $\color{red}{\blacktriangle}$ +18.41 13.94 $\color{red}{\blacktriangle}$ +13.27 40/217
agentless 39.98 13.94 $\color{red}{\blacktriangle}$ +26.04 13.94 $\color{red}{\blacktriangle}$ +26.04 40/217
agentlessCodeCoverage 114.53 84.36 $\color{red}{\blacktriangle}$ +30.17 82.69 $\color{red}{\blacktriangle}$ +31.84 40/217
agentlessLineCoverage 101.94 118.53 $\color{green}{\blacktriangledown}$ -16.59 123.36 $\color{green}{\blacktriangledown}$ -21.42 40/217

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@calvinbayer
calvinbayer marked this pull request as ready for review October 19, 2026 22:10
@calvinbayer
calvinbayer requested review from a team as code owners October 19, 2026 22:10
@calvinbayer
calvinbayer requested review from ValentinZakharov and removed request for a team October 19, 2026 22:10
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T22:13:36.619184Z 0b80b18 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

JUnit 5 skip selection receives declaration metadata consistently with the existing retry-policy and reporting paths. The completed static review identified no actionable regression against the supplied base.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 0b80b18

@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from 0b80b18 to 9e1ec7c Compare October 19, 2026 23:01
@calvinbayer
calvinbayer requested a review from a team as a code owner October 20, 2026 08:09
@calvinbayer
calvinbayer requested review from sarahchen6 and removed request for a team October 20, 2026 08:09
@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from 9e1ec7c to 73c9c70 Compare October 20, 2026 08:09

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

The new exclusion resolver lacks coverage in internal-api’s own JaCoCo report, causing the required Java 8 coverage gate to fail despite tests in dependent modules.

Open Bits AI session

🤖 Bits Code Review · Commit 9e1ec7c


Findings that could not be posted inline

internal-api/src/main/java/datadog/trace/api/civisibility/config/TestExclusions.java:12

P2 Add owning-module coverage for the exclusion resolver

Java 8 CI runs baseTest with -PcheckCoverage, enforcing internal-api’s per-class minimums of 70% branch and 80% instruction coverage. This new helper has no module-local tests or coverage exclusion; tests added in other modules do not contribute to its report. The required coverage verification therefore fails. Add internal-api tests for the resolver and cover or explicitly exclude the newly executable TestEventsHandler.skipReason default method.

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🤖 Bits Code Review · Open Bits AI session

@calvinbayer

Copy link
Copy Markdown
Author

Addressed the coverage finding in this AutoTest review with 8028762, which this PR inherits from its JUnit4 parent.

The owning-module JaCoCo gate failed before the test additions: TestExclusions and the new default handler callbacks had zero coverage. Four durable behavior cases now cover independent annotations, tag resolution, empty/unknown metadata, and compatibility delegation through the legacy callbacks. On current master, the complete internal-api suite passes 5,256 tests and the coverage gate passes with 100% instruction and branch coverage for TestExclusions and 100% instruction coverage for the added TestEventsHandler defaults. No coverage thresholds were lowered.

@calvinbayer

Copy link
Copy Markdown
Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The JUnit 5 skip decision now receives declaration metadata and tags consistently with the existing retry-policy and reporting paths. The completed static review identified no actionable regression.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 73c9c70

@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from 73c9c70 to 2d9d2d2 Compare October 20, 2026 18:53
@calvinbayer

Copy link
Copy Markdown
Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Method annotations and JUnit tags reach the exclusion-aware skip policy, consistent with the metadata already used for retries and reporting. Native skips and developer test selection retain their existing semantics.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 2d9d2d2

@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from 2d9d2d2 to db4f4d4 Compare October 28, 2026 09:36
@calvinbayer

Copy link
Copy Markdown
Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

JUnit 5 supplies test-source metadata and tags to the existing exclusion-aware skip policy, enabling declaration-level protections during skip decisions. Runtime behavior remains unverified because tests were not executed.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit db4f4d4

@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from db4f4d4 to 3cc0ab4 Compare October 28, 2026 11:31
@calvinbayer

Copy link
Copy Markdown
Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

JUnit5 skip decisions now receive the declaration metadata needed to honor automation exclusions, consistent with execution policies and reporting. The completed specialist review identified no actionable regression in the supplied diff.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 3cc0ab4

@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from 3cc0ab4 to 16de1ed Compare October 28, 2026 15:47
@calvinbayer

Copy link
Copy Markdown
Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

JUnit5 skip-policy selection receives declaration metadata while retaining the ITR unskippable guard. Static review identified no actionable defect; runtime behavior remains unverified because tests were inspected but not executed.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 16de1ed

@calvinbayer
calvinbayer force-pushed the codex/junit5-automation-exclusions branch from 16de1ed to 308e8e1 Compare October 28, 2026 16:40
@calvinbayer

Copy link
Copy Markdown
Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Declaration metadata reaches the shared exclusion policy while preserving framework skips and ITR unskippable handling. Static inspection found no reportable defect in the JUnit 5 changes.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 308e8e1

@calvinbayer

Copy link
Copy Markdown
Author

Shepherd verification — JUnit5 native tests and packaged Java smoke

Tested by: Shepherd Agent (autonomous QA for Datadog Test Optimization)
Revision tested: 308e8e188034f181b164eedc1cd8e2dba11cba60
Environment: Local macOS arm64, JDK 21, Mockdog.

  • Six native JUnit5 exclusion/TIA cases passed, including the unskippable paths; formatting passed.
  • All 35 Jolokia default assertions passed through the normal Crook workflow with --debug and the pinned local agent JAR.
  • Recorded 1,769 events: 1,404 TestNG tests, 188 suites, 32 modules, one session.
  • All 17 embedded version resources and every observed runtime version match 308e8e1880.
  • Agent SHA256: c1d53009ecee391e07d50266fdf61bf84cd7c18037502790838c4945b9caf1a8.
  • Source worktree remains clean; the log records dependency restoration and owned Mockdog shutdown.

Jolokia is a TestNG packaged-agent smoke, not JUnit5 workload coverage. Native JUnit5 instrumentation tests verify the changed JUnit5 paths. This run uses Mockdog; no Datadog UI or production-backend validation is claimed. No backend API request/response changes were made.

Issues found: No additional JUnit5 correctness issue was found. The inherited JUnit4 parameter-label regression was fixed in 855c0da96c. Cached development-version resources were regenerated through normal Gradle tasks before the final smoke run; no source or binary patch was used.

@calvinbayer
calvinbayer added this pull request to stack #13114 October 28, 2026 23:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants